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Abstract 

The purpose of this grounded theory study was to identify the 
typical barriers encountered by faculty and administrators when 
implementing outcomes-based assessment program review. An 
analysis of interviews with faculty and administrators at nine 
institutions revealed a theory that faculty and administrators’ 
promotion, tenure (if applicable), and/or renewal of contracts 
are often not dependent on whether they use results front 
outcomes-based assessment program review to improve their 
students’ learning and development. 


IDENTIFYING BARRIERS IN 
IMPLEMENTING OUTCOMES-BASED 
ASSESSMENT PROGRAM REVIEW: 

A GROUNDED THEORY ANALYSIS 


While conversations proposing standardized testing within higher education 
abound (Allen & Bresciani, 2003; Department of Education (DOE), 2006; Ewell, 
1997a, 1997b; Ewell & Jones, 1996; Maki, 2004; Palomba &Banta, 1999), proponents of 
outcomes-based assessment program review are still applauding the value and extent that 
the process can be used to inform decisions to improve student learning and development 
(Bresciani, 2006; Bresciani, Zelna, & Anderson, 2004; Huba & Freed, 2000; Maki, 
2004; Mentkowski, 2000; Palomba & Banta, 1999; Suskie, 2004). As such, practitioners 
of outcomes-based assessment continue to seek various ways to meaningfully engage in 
outcomes-based assessment program review in order to find ways to improve student 
learning and development. 

Even so, there are many others who do not believe the process adds value to their 
day-to-day teaching or administrative duties (Banta, 2002; Wergin, 1999). Regardless 
of whether individuals agree upon the level of value that engaging in outcomes-based 
assessment may bring to improving student learning and development, many instructors, 
administrators, and scholars still experience barriers in the implementation of the process. 

The purpose of this grounded theory study was to identify the typical barriers 
encountered by faculty and administrators at a variety of institutional types when 
implementing outcomes-based assessment program review. While the intent of the 
study was to identify barriers and explore strategies that institutions use to address those 
barriers, a theory emerged as to why the practice of outcomes-based assessment is not 
pervasive even in institutions whose leadership emphasizes the importance of such a 
process to improve student development and learning. 


Literature Review 
The Common Institutional Barriers 

Research has been conducted to illustrate the common barriers to implementing 
outcomes-based assessment. The reasons that outcomes-based assessment is not 
pervasively practiced or practiced at all are often classified into three categories: (a) 
time, (b) resources, and (c) understanding of assessment (Banta, 2002; Bresciani, 2006; 
Bresciani et al., 2004; Palomba & Banta, 1999; Suskie, 2004; Upcraft & Schuh, 1996). 
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As these categories are dissected further, additional reasons are revealed. 

Time. Research posits that the manner in which one allocates time is influenced 
by how one prioritizes one’s values (Argyris & Schon, 1978; Argyris & Schon, 1996; 
Dalton, Healy, & Moore, 1985; Sandeen, 1985). As such, human beings, regardless of 
their profession, will allocate their time that is devoted to work based on what they value 
or what they are told to value by those responsible for evaluating job performance (Argyris 
& Schon, 1978; Argyris & Schon, 1996; Bass, 1985; Bass & Avolio, 1994). Furthermore, 
while it can be assumed that all people have been given the same amount of time, all do 
not have the same number of priorities or level of responsibilities pressing upon their time. 
Thus, it is uncertain as to how decisions are made in accordance to varying number of 
priorities or responsibilities that press upon the amount of time that each person has to 
spend. 

Certainly, in addition to personal and professional values, personality styles and time 
management strategies play a role in how people choose to prioritize their work projects in 
the time that they have allocated to their profession (Hackman, 1990; Kirkpatrick, 1993). 
The manner in which a person is evaluated and the criteria applied to personnel review 
also may influence how persons allocate their time at work (Hackman, 1990; Petrini & 
Hultman, 1995). 

Resources. Resources have been presented as a reason that people do not engage 
in outcomes-based assessment including (a) the cost of providing professional development 
to faculty and administrators in order for them to learn how to engage in quality outcomes- 
based assessment, (b) the cost of the time re-allocated from actually teaching to the evaluation 
of teaching or from delivering the program to its evaluation, and (c) the cost of providing 
retreats so that faculty and administrators can actually reflect on what the outcomes-based 
assessment data are telling them about their program or curriculum. In addition, there is 
the cost of the administration and analysis of the evaluation tools used in outcomes-based 
assessment, as well as the cost of the improvements recommended for the program as the 
data suggests (Bresciani, 2006; Bresciani et al., 2004; Palomba & Banta, 1999; Suskie, 2004; 
Upcraft & Schuh, 1996). 

The actual costs of implementing outcomes-based assessment often go uncalculated. 
In an attempt to determine actual costs of engaging in outcomes-based assessment, or rather, 
the attempt to evaluate the evidence of student learning and development, administrators 
are unsure as to whether to place the costs in the instructional category, an institutional 
research category, or an unfunded mandate category (Addison, Bresciani, & Bowman, 
2005; Bresciani, 2006). Furthermore, the start-up costs of educating personnel to learn 
how to implement effective, efficient, and enduring outcomes-based assessment are often 
never allocated (Palomba & Banta, 1999; Upcraft & Schuh, 1996). Because the actual 
cost of engaging in outcomes-based assessment has not been systematically calculated, it is 
difficult to determine whether the perceived or actual costs of professional development are 
off-set by improved student learning. 

Understanding of assessment. Higher education apparently has been fraught 
with “flavor of the day” processes and reporting initiatives, and institutional memory is 
often long about these fads (Banta, 2002; Palomba & Banta, 1999; Petrini & Hultman, 
1995; Upcraft & Schuh, 1996). As such, faculty and administrators are often wary of 
anything else that comes along in an apparently pre-packaged version or with the threat of 
an unfunded mandate. While outcomes-based assessment has been around in one form 
or another for quite some time (Banta, 2002; Bresciani, 2006; Palomba & Banta, 1999; 
Upcraft & Schuh, 1996), the assumption that it is really here to stay is understandably 
questioned because the manner in which outcomes-based assessment has been labeled has 
changed over the years. 

Many administrators and faculty simply do not believe that outcomes-based 
assessment is designed to be a systematic process to improve student learning and 
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development, rather than a process to sustain itself (Bresciani et ah, 2004; Kuh, Kinzie, 
Schuh, & Whitt, 2005; Upcraft & Schuh, 1996). In addition, the increasing emphasis 
on accountability, using standardized testing and other performance indicators that 
often cannot be linked to what is actually occurring in the classroom or within the co- 
curricular causes further understandable confusion. 

In reviewing the three primary published reasons that faculty and administrators 
do not engage in outcomes-based assessment, a question arises: If an institution claims 
in its mission statement that it values student learning and development, why is it that 
that statement does not translate into action such as an institutional expectation for the 
evaluation of how well that learning and development is done in a manner in which it 
can be improved? 

Outcomes-based assessment has several definitions (Palomba & Banta, 1999). 
Regardless of which definition one is examining, the idea of continuous improvement 
is often a common element (Allen, 2004; Banta, 2002; Bresciani et ah, 2004; Maki, 
2004; Palomba & Banta, 1999; Suskie, 2004). Using continuous improvement in the 
definition, there is an assumption of purposeful planning for the delivery and evaluation 
of intended outcomes. In addition, the evaluation process is designed so the information 
gathered could be used to inform specific decisions about how the intended outcomes 
can be met at a greater level of quality for the group that was included in the evaluation. 

There are several resources designed to assist faculty and administrators with 
implementation of outcomes-based assessment (Allen, 2004; Bresciani, 2006; Bresciani 
et ah, 2004; Maki, 2004; Palomba & Banta, 1999; Suskie, 2004; Upcraft & Schuh, 1996), 
yet many faculty and administrators are still having difficulty meaningfully engaging in 
the process. Why is that? 

The purpose of the grounded theory study was to identify the typical barriers 
encountered by faculty and administrators at a variety of institutional types when 
implementing outcomes-based assessment program review. While the intent of the 
study was to examine barriers and explore strategies that institutions use to address those 
barriers, a theory emerged as to why the practice of outcomes-based assessment may not 
be pervasive even in institutions whose leadership emphasizes the importance of such a 
process to improve student development and learning. 


Methodology 

To better understand how faculty and administrators are challenged with 
implementing outcomes-based assessment program review, a qualitative method of 
inquiry was utilized because the researcher’s intent was to uncover rich and descriptive 
“meaning” (Bogdan & Biklen, 1992). There are several methods by which meaning 
can emerge and many of them share the common goal of understanding the subject’s 
perspective. Researchers using grounded theory attempt to generate a theory that is 
closely related to the context of that which is being studied (Strauss & Corbin, 1990). 

In grounded theory, the researcher interviews subjects and examines documents, 
then returns to evaluate the transcripts and documents using open coding analysis 
in order to identify categories or properties about what is being studied. Next, the 
researcher returns to the field to interview more subjects, continues with open coding, 
but also begins to use axial coding to compare the interviews in order to understand 
the central phenomenon, such as the reason that faculty and administrators are not 
pervasively engaging in outcomes-based assessment. Axial coding involves the act of 
constantly comparing words and meanings in order to formulate some common themes 
across the data. In the case of this study, the purpose of axial coding is to identify 
categories or conditions that may be contributing to the subjects’ inability to engage in 
outcomes-based assessment and to identify specific strategies, conditions, and contexts 
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that influence practice or in this case, the student affairs/services professionals’ willingness 
to practice outcomes-based assessment. In the third stage of data analysis, selective coding, 
the researcher uses the results of open and axial coding to integrate categories in order to 
identify a theory that further explains the complexities of the research findings (Creswell, 
1998). 


Research Questions 

In an effort to explore how pervasive faculty and administrators were engaged in 
outcomes-based assessment program review, the following research questions guided the 
analysis of the case studies and interviews. 

1. How pervasively are your faculty and administrators engaged in outcomes- 
based assessment program review? 

2. What do they report as challenges in their ability to meaningfully and 
manageably engage in the process? 

3. How do they address those challenges? 


Selection of Sample 

Nine institutions were purposefully selected to participate in this study. All nine 
institutions were considered to be good practice or emerging good practice institutions 
in implementing outcomes-based assessment program review based on good practice 
criteria published by Bresciani (2006). There were three community colleges, three 
comprehensive institutions, and three research extensive universities. At least three faculty 
and three administrators were interviewed at each institution. At some institutions, 
due to the opportunity provided to the researcher, more faculty and administrators were 
interviewed. In addition, documents, such as personnel evaluation criteria and documents 
(when available), meeting minutes, faculty memos, and institutional websites were also 
reviewed. 


Limitations 

Limitations of this study include the inability to verify among decision makers the 
degree to which contributions or improvements to student learning is factored into personnel 
evaluations. In many instances, criteria for the weight that is placed on such an evaluation 
could not be ascertained in faculty evaluations nor in evaluations of administrators. In 
addition, it was unclear, apart from the use of student evaluations of teaching effectiveness, 
what criteria for improvement or contributions to student learning and development 
were being used for faculty evaluations. Similarly, there was little evidence of criteria for 
contributions to student development being used for administrative evaluations. 

Another potential limitation is that those who participated in the interviews 
were motivated to illustrate a positive or negative aspect of the extent that faculty and 
administrators are using outcomes-based assessment results to improve student learning 
and development at their institution. 


“If an institution 
claims in its mission 
statement that 
it values student 
learning and 
development, 
why is it that that 
statement does not 
translate into action 
such as institutional 
expectation for the 
evaluation of how 
well that learning 
and development 
is done in a manner 
in which it can be 
improved?” 


Findings and Discussion 

In an effort to explore how pervasively faculty and administrators were engaged in 
outcomes-based assessment program review, the aforementioned research questions guided 
the grounded theory analysis of the case studies and interviews. 

A grounded theory analysis of the interviews and documents revealed a theory as to 
why faculty and administrators’ engagement in outcomes-based assessment program review 
may not be pervasive. Faculty and administrators’ promotion, tenure (if applicable), and/or 
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renewal of contracts is often not dependent on whether they use results from outcomes- 
based assessment program review to improve their students’ learning and development. 
Rather, the promotion, tenure (if applicable), and/or renewal of their contracts appears 
to be based on a level of productivity that evidence suggests may not directly be related to 
improved student learning and development. Dependent on institutional type, reviews 
appear to be focused on either (a) number of peer-reviewed research journals published, 
(b) number and funding level of grants received, (c) in-class instructional evaluations by 
students, and (d) level of outreach activity, and/or other performance activities pertaining 
to increased inputs or outputs that improve performance indicators used for funding but 
are not necessarily related to student learning and development. To further illustrate the 
emergence of this theory, the findings are presented and discussed under each research 
question. 


Pervasive Involvement in Outcomes-based Assessment Program Review 

In regards to how pervasively faculty and administrators are engaged in outcomes- 
based assessment program review, the institutions that had leadership commitment to 
evidence-based decision making had faculty and administrators in every unit of their 
organization engaged in some level of systematic outcomes-based assessment program 
review. When asked how this came to be, one administrator’s response was illustrative 
ot the many others when he said: 

We didn’t arrive at this level of involvement overnight. It took years of 
consistent commitment to building a process that faculty would recognize as 
first and foremost meaningful to them. They had to see that improvements 
[in student learning and development] would be made and that they were 
improvements that mattered to them and what they wanted students to learn. 

While faculty, academic support specialists, and student support specialists were 
necessary to the establishment of a process that generated useful data to informing 
decisions that led to improvement, not all faculty and administrators were engaged in 
that process. For the most part, those that engaged in the process did so because they 
found the systematic practice of outcomes-based assessment valuable to improving 
student learning and development. 

Many of these faculty and administrators reported that they had been 
implementing the process, albeit informally, long before the leadership called for all to 
be involved in it. As one participant stated, “I had been doing this [outcomes-based 
assessment] for years. I just didn’t know it was called that [outcomes-based assessment].” 
Another illustrated the thought by sharing: 

We [faculty] have discussions about how to improve student performance all of 
the time. We have them around the coffee pot; we have them in department 
meetings; we have them when we are standing in the halls; and we have them 
right after a student we are concerned about leaves our office. We care about 
improving student learning because we care, not because someone told us we 
had to care. 

Such comments were common among the faculty represented in the institutions 
where outcomes-based assessment practice was prevalent. When asked why this was the 
case, one administrator summarized a common response among many participants when 
she said: 

The faculty who are engaged in outcomes-based assessment in a systematic way 
would have done it anyway. These faculty experiment with inquiry processes 
to improve student learning because they want their students to improve. All 
we [administration] did was help them systematize it, provide some support so 
they could structure the process into program review and offer some release 
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time and professional development to get other faculty to discover its benefits. 

Our faculty don’t do it because they are externally [external to the academy] 
or internally [internal to the academy] rewarded for doing it [outcomes-based 
assessment]. They do it [outcomes-based assessment] because it provides them 
with specific information to inform decisions that will improve student learning. 

Huba and Freed (2000), Maki (2004), Mentkowski and associates (2000), Suskie 
(2004), and Palomba and Banta (1999) have repeatedly illustrated that the primary 
motivation for faculty to engage in systematic outcomes-based assessment comes from the 
realization they discover, after trying it out, that it does indeed contribute to improved 
student learning and development. So, when one examines how pervasive faculty 
involvement is in outcomes-based assessment, does one assume that if faculty experience it 
and recognize its value that they will then systematically implement it? If so, how do you 
get the faculty and administrators who are not involved to become involved? As reported 
by one administrator: 

I am not sure which comes first. It is the proverbial chicken and the egg. While 
well-respected faculty have designed the process and to some extent, I think they 
have peer-pressured other faculty into getting involved... [pause] Still, not every 
faculty member is involved. And I don’t think they have to be, even though we 
want them to be. 

While administrators and faculty commented on whether they felt the practice of 
outcomes-based assessment program review on their campuses was pervasive, there was no 
clear definition as to what pervasive meant and how many faculty or administrators made 
the practice “pervasive.” 

In the Merriam —Webster dictionary, pervasive means, “pervading or tending 
to pervade” (“Pervasive,” 2007). If seeking the definition of pervading or pervade, you 
would discover that it means, “to become diffused throughout every part of’ (“Pervading,” 
2007). If pervasive means getting every faculty member and administrator involved in 
outcomes-based assessment, then when it comes to how to get faculty involved, several 
ideas were shared by study participants. Many of the ideas shared range from hiring faculty 
and administrators who are able to do outcomes-based assessment as advertised for in 
position descriptions, to providing release time to engage in outcomes-based assessment, 
to providing other rewards and incentives for involvement, and clarifying expectations for 
involvement (Bresciani, 2006; Bresciani et ah, 2004; Huba & Freed, 2000; Maki, 2004; 
Mentkowski, 2000; Palomba & Banta, 1999; Suskie, 2004). 

Administrators desiring for their staff to become more involved in outcomes-based 
assessment simply stated that, “getting them [administrative staff] involved? That is easy. 
They are not faculty; you can just tell them [administrators], it has to be done and they do it. 
They [administrators] can’t hide behind academic freedom.” Even with such confidence, 
it was clear that not every administrator was engaged in outcomes-based assessment, even 
when the administrative leadership felt they had made the expectations for such involvement 
very clear. 

If pervasive does not constitute getting every person involved in outcomes-based 
assessment but ensuring that at least someone in every aspect of the organization is engaged 
in outcomes-based assessment, how does one still manage to get people who are not 
currently involved to participate in the process? 


“...the fact of the 
matter is that the 
only way I can 
keep my job...the 
only way I can 
keep teaching is if 
I publish several 
articles in journals 
that only accept less 
than io percent of 
the submissions. I 
tried to explain this 
to a student once... 
when I could not see 
them because I had 
to get my research 
done.” 


Challenges to Engaging in the Process 

Participants in this study reiterated the common challenges to engaging in 
outcomes-based assessment: (a) time, (b) resources, and (c) understanding of assessment. 
For brevity’s sake, the researcher will not expand on these challenges since they are re¬ 
affirmed in several publications (Bresciani, 2006; Bresciani et ah, 2004; Huba & Freed, 
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2000; Maki, 2004; Mentkowski, 2000; Palomba & Banta, 1999; Suskie, 2004). What 
may be more compelling at this point is to discuss the barrier that participants, regardless 
of institutional type, felt was most difficult to address. This barrier is as follows. 

Faculty and administrators’ promotion, tenure (if applicable), and/or renewal 
of contracts is often not dependent on whether they use results from outcomes-based 
assessment program review to improve their students’ learning and development. 
Rather, the promotion, tenure (if applicable), and/or renewal of their contracts appears 
to be based on a level of productivity that evidence suggests may not directly be related 
to improved student learning and development. Therefore it appears that the evaluation 
of personnel processes, particularly for faculty, dissuade faculty in engaging in extensive 
evaluation of student learning and development. As this faculty member illustrates: 

Listen, it is not that I don’t care about what my students learn and how well 
they learn it; I do. I really care. But the fact of the matter is that the only way 
I can keep my job... the only way I can keep teaching is if I publish several 
articles in journals that only accept less than 10 percent of the submissions. I 
tried to explain this to a student once... when I couldn’t see them because 
I had to get my research done... [pause] I stopped myself [from telling the 
student this]. How do you explain that to a student who needs your help? 

Several tenured, full professors, who are actively engaged in outcomes-based 
assessment on their campuses and who would be described by their senior administrators 
as the faculty who led the design and implementation of the process that systematically 
improved student learning affirmed the rewards process for publications. “You can’t 
have junior (untenured) faculty getting involved in documenting how well their students 
learn. You just can’t. It takes too much time away from their research.” These faculty 
expressed time and time again how responsible they were being by “protecting” assistant 
professors, so they could get their tenure. “They need to get their tenure, and then we 
will ask them to assess [student learning]”, explained a full professor. 

Faculty at less research intensive institutions felt the same pressure to publish 
first, and then to consider the evaluation of student learning second. 

I hate it when people assume we don’t care about student learning; that is not 
the case. It’s just that I have seen some teachers go untenured [pause]... the 
ones who only talk about student learning and measuring how well they do 
it.. .if you don’t pay attention to your research, you don’t get to stay. 

In two-year colleges where there is little to no research pressure, this concern is 
expressed slightly differently. 

No, we don’t have the research pressures that others have, but we get less time 
to plan our preps than the high school teachers get. Also, there is no time to 
sit and chat about we learned from our classroom assessment. I am gathering 
the data [about how our students learn], but we are so busy teaching, we have 
no time to talk about what our students are learning. 

When these two-year college faculty were asked about the possibility of release 
time to reflect on student learning results in order to inform conversations where decisions 
could be made for improvements, some mentioned that their collective bargaining 
agreements were being interpreted by their union leadership in a manner that would 
dissuade them from doing this: 

I tried to get a group of faculty together where we would talk about what 
we were finding out in our classrooms. I was visited by a union leader who 
discouraged me from doing this, telling me that this type of work was outside 
the scope of our collective bargaining agreement. I was confused by that 
remark but I haven’t had time to look into it further. 

This faculty member, along with others in the study, understood that hosting 
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such conversations was not in the scope of their duties and they would need additional 
compensation and possibly an additional contract to have such conversations. 

Other two-year faculty members engaged in outcomes-based assessment reported 
that they are doing it as additional work for which they are neither recognized nor 
reimbursed: 

If you look at how our workload is calculated, there is zero time allocated to the 
practice of outcomes-based assessment. We [my colleagues and I at other 2-year 
institutions] get evaluated based on how many FTEs [Full-Time Equivalent], 
SCHs [Scheduled Class Hours], seat hours, or continuing education hours we 
generate. That is what we hear about; that is what we understand we are to care 
about.. .just productivity of increased numbers. How can you find time to focus 
on student learning when that is all you hear? 

Even though faculty of all institutional types may have painted a bleak picture, 
many of them also re-affirmed that regardless of whether they get evaluated for their 
contributions to student learning, they will do so, and they will use the data to discuss ideas 
for improvement with their colleagues. When asked how they manage to evaluate student 
learning and use the results, many responded with similar answers as this: 

I work my ass off. I can’t fall behind on publications. It doesn’t matter what 
level [of professor] you are around here. The higher you go [in promotion and 
tenure], the more work you get; the more responsibility you have. Somebody 
should do a study of divorce rates and loss of custody of children among faculty 
and why it occurs. Now, that would be something to study. 

Some faculty were able to use the results of their outcomes-based assessment work 
for publications and thus, were able to “kill two birds with one stone.” But many more 
faculty said that the evaluation of their students’ learning was not anywhere close to what 
their faculty peers would value as research in their discipline. 

For many administrators, continuance of employment may be perceived as political 
or based on countable production and activity, rather than evidence of contributions to 
student learning and development: 

Look at how I evaluate people, there is nothing in there asking them to 
show evidence of their organization’s contribution to student learning or 
development.. .nothing. We do it anyway though, because we know it is what 
we are all about. 

One other senior level administrator however, made his expectation very clear, “If 
you don’t tell me how you know you are contributing to student learning, I will tell you 
that you just made you, and your department candidates for ‘outsourcing.’” While this 
administrator made his expectations very clear, there was no clear evidence that his staff was 
evaluated and rewarded for the extent they could demonstrate improvements in student 
learning and development. 

Scholars have written about the notion that lasting change cannot occur in higher 
education unless the rewards structure for making that change also follows suit (Banta, 
2002; Cox & Richlin, 2004; Doherty, Riordan, & Roth, 2002; Eckel, Green, & Hill, 2001; 
Frazier & Frazier, 1997; Hutchings, 2001; Kreber, 2001; Maki, 2004). Given this line of 
research and the findings in this study, one may wonder if the institutional leadership who 
are committed to improving student learning and development can sustain the efforts if 
their personnel evaluation systems are not updated to reflect a change in organizational 
values. 


Addressing the Challenges 

Apart from being able to arrive at a strategy to address the barrier that personnel 
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are not rewarded for the extent that they use evidence to improve student learning 
and development, the institutional expectation for improving student learning and 
development was still made clear at the institutions that participated in this study. As 
such, they were eager to share their solutions for addressing the three common barriers 
identified: (a) time, (b) understanding of assessment, and (c) resources. 

Time. As previously mentioned, many participants discussed the fact that there 
was no new time to allocate to the reflection and improvement of student learning and 
development. Participants reiterated that time either had to be re-allocated from their 
personal time or from some other projects. As one administrator stated: 

Everyone cares about students here and we are a small institution so many of us 
carry a lot of responsibilities. In order to get this [outcomes-based assessment] 
done, we just get it done. We work harder and smarter. That is just the way 
it has to be. 

A faculty member at another institution represented a different approach: 

We know that it [outcomes-based assessment] won’t get done well if we don’t 
re-allocate time to it. So, that is what we do. We have invested a lot of time 
in learning how to do this well. We have re-designed our faculty meetings 
so that we discuss the results and apply what we learned. We are not able to 
respond to all of the needs we see; we simply can’t. But we do move forward 
with improvements every year. We are very proud of that. 

Some common strategies discussed by most of the participants in this study included the 
following, (a) Re-allocation of time from the doing of the activity to the evaluation of the 
doing. This may mean investing in release-time from activities for both administrators 
and for faculty, (b) When an organization cannot provide release time, encourage faculty 
and administrators to engage in fewer activities so that they can reflect on the data and 
decisions that need to be made in order to improve learning and development, (c) Discuss 
results derived from outcomes-based assessment in a manner that is open and inclusive, 
the practice of which saves time when people are wondering how and why decisions are 
made, (d) Collaborate with peers to assess student learning, discuss the results, and make 
decisions. If you share your workload, which includes involving students in every aspect 
of the process, you save time because you are borrowing ideas from colleagues. They 
may be able to suggest solutions more quickly because they are not as invested in the 
history of what led you to do that which you now do and have recently discovered is not 
as an effective process as you had hoped. 

Understanding of assessment. Similar to previous research findings, the need 
to understand what outcomes-based assessment is and why one should engage in it was 
prevalent (Bresciani, 2006; Bresciani et al., 2004; Huba & Freed, 2000; Maki, 2004; 
Mentkowski, 2000; Palomba & Banta, 1999; Suskie, 2004). In particular, participants 
spoke of the importance of balance in understanding how results would be used and the 
level of expectations for engagement in assessment among faculty, administrators and top 
level leadership: 

It is a delicate balance. You need the expectations communicated from 
the leadership that this is a process that will help us systematically improve 
students’ learning, but it needs to be communicated in a manner that allows 
faculty the freedom to develop the process that is most meaningful to them; a 
process where they can discover whether the results will actually be helpful in 
improving learning. 

Another participant explained: 

To keep this (outcomes-based assessment) from being seen as an unfunded 
mandate, we made sure to connect the required documentation to what we 
already were doing with planning and program review. Doing this was so 
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helpful to everyone involved; they could focus on the meaning of doing it, rather 
than griping about having to do it. 

Participants illustrated the importance of making sure everyone, including top-level 
leadership could understand (a) what outcomes-based assessment is and why it is being 
required, (b) the connection of assessment to planning, (c) how to use the results both as a 
practitioner (e.g., administrator or instructor) and as a leader, and (d) how to connect the 
results to external benchmarks or indicators of success. However, in order to understand 
how to do all this, one participant remarked: 

You really need to provide training or education or whatever you want to call 
it. You can’t just expect that faculty and administrators are going to simply 
understand this just because you have shown them evidence of how the process 
can improve student learning. You really have to educate .. .and that takes time... 

A commitment to providing professional development takes time and it takes resources 
in the manner of investing in the professional development (Bresciani, 2006; Bresciani et 
al., 2004; Huba & Freed, 2000; Maki, 2004; Mentkowski, 2000; Palomba & Banta, 1999; 
Suskie, 2004). 

Resources. While time can certainly be considered a resource, this section focuses 
on the types of resources that could be provided to faculty and administrators such as 
professional development to learn all aspects of the outcomes-based assessment process, 
including follow-up one-on-one assistance, assistance with documentation, and facilitated 
reflection for interpretation of results and decision-making. Most of the participants agreed 
that early investment in professional development was needed in order for the process to be 
sustainable and effective: 

We invested a great deal of money in up front professional development. When 
we decided to do this, we actually followed a corporate model for ‘re-tooling’ 
our workforce. Now, since we have done this for a while, we really have 
more of a ‘train-the-trainer’ model so the departments absorb the professional 
development costs as they ‘orient’ faculty and staff to the way they do things. 

Many scholars (Battino et al., 2006; Eddy, 2005; Kemp et al., 2006; Stanley, Watson, 
& Algert, 2005) discuss the importance of investing in faculty training and development 
in order to “re-tool the workforce” for changes in the industry such as ways to evaluate 
and improve student learning and development. These scholars, while not fully adopting a 
corporate model explain the importance of realizing that needs in higher education change, 
society changes, and students change; thus, to not invest in the re-equipping of our faculty 
and staff is irresponsible and short-sighted. 

While the participants in this study found varying ways to provide and fund 
professional development, they all did so and continue to do so in the ways that make the 
most sense to each institution. However, the documentation of the process and the results 
and decisions still posed challenges. One participant illustrated this when she said: 

I do care about how well my students learn and develop, but outcomes-based 
assessment takes a great deal of time to document. We can do this easily, but 
documenting it all? [pause] I am just not sure that the improvements that result 
from engaging in it are worth the time [invested in it]. 

When pressed for an explanation, this faculty member returned to the earlier expressed 
concern that what she really needed to document were the items that went into her 
promotion reviews; how she used outcomes-based assessment to improve student learning 
was not required in that paperwork. 

Recommendations 


“You can’t just 
expect that faculty 
and administrators 
are going to simply 
understand this 
just because you 
have shown them 
evidence of how the 
process can improve 
student learning. 
You really have to 
educate...and that 
takes time...” 


Clearly, more research needs to be conducted in order to determine the extent that 
faculty and administrators are not reviewed and promoted or tenured (if applicable) based 
on their contributions to student learning and development. Data are inconclusive. 
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Even so, it should be noted that when reviewing all personnel evaluation guidelines, none 
of these institutional processes asked for faculty and administrators to provide evidence, 
based on a systematic evaluation process, as to how student learning and development 
was improved. Furthermore, while faculty and administrators recognized that this 
information was not provided, there appeared to be very little discussion around how 
to change existing processes. Further research on why that may be the case would aid 
additional conversations on the matter. 

Given that faculty and administrators may not be evaluated and subsequently 
rewarded for improving student learning and development, it appears that regardless of 
what tool (e.g., CLA, ETS) or what process (e.g., outcomes-based assessment, CQI) is 
used to evaluate student learning and development, pervasive improvement in student 
learning and development may not come about if faculty and administrators’ promotion 
and review processes are not influenced by the use of outcomes-based assessment for 
improvement. On the contrary, if faculty and administrators responsible for designing 
learning and development activities are told what tools and what processes they have to 
use without being able to develop these tools and measurements themselves, it may be 
likely that given these findings, there will be even less motivation and subsequent reward 
to improving student learning and development. Certainly there may be less clarity in 
regards to how to specifically improve student learning (Maki, 2004; Suskie, 2004). 
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